The standard approach to presenting information in a meeting, classroom, or similar context is via a computer coupled to a projector. The presenter loads a slide deck or other teaching material onto the computer, and then uses the projector as an output device to project the video output of the computer onto a screen or wall. In this approach, it is difficult for the presenter to interact with the teaching materials, beyond paging through the slide deck or other simple interaction. For example, if the presenter wishes to manipulate objects shown on the screen, the presenter needs to use a mouse or other pointer input device. Unfortunately, using a mouse during a presentation is often awkward; for example, because it may be difficult for the presenter to simultaneously use the mouse, interact with the audience, and see the screen.
One approach to addressing these issues is to instead use a touch-enabled display. One type of touch-enabled display is an active touch-sensitive flat panel video display, such as a touch-enabled LCD monitor. Another type of touch-enabled display is a projection-based system which includes a touch-sensitive panel upon which video images are displayed by way of a projector. One example of such a projection-based system is the SMART board produced by Smart Technologies.
Unfortunately, such touch-enabled displays suffer from a number of drawbacks. First, touch-sensitive flat panel LCD displays can be quite expensive, particularly when larger display sizes are desired. Also, technologies such as the SMART board require special hardware, in the form of a touch-sensitive white board. Also, such technologies are not capable of detecting the position of the user's finger or hand unless it is in contact with the display surface. In other words, these technologies only function in two dimensions, limiting the types of user interface modalities they can support.
The arrival of gesture control devices including Microsoft Kinect, Intel RealSense, Leap Motion and some other wearable devices such as the Myo arm band has given rise to a new mode of human computer interaction using gestures in the air. While such devices do have a three dimensional aspect to their interaction, the user often has the complex task of coordinating their air gestures with the interface that is set at a distance. Using such devices, the user does not have a direct, tangible interaction with a physical object (e.g., a screen) that provides a “what you touch is what you are interacting with” type experience. This shortcoming often results in frustration and/or extensive training of the user.